Polynomial Networks and Factorization Machines: New Insights and Efficient Training Algorithms

Supplementary material A. Symmetric tensors A.1. Background Let Rd1⇥···⇥dm be the set of d1⇥ · · ·⇥dm real m-order tensors. In this paper, we focus on cubical tensors, i.e., d1 = · · · = dm = d. We denote the set of m-order cubical tensors by Rd m . We denote the elements of M 2 Rdm by Mj1,...,jm , where j1, . . . , jm 2 [d]. Let = [ 1, . . . , m] be a permutation of {1, . . . ,m}. Given M 2 Rdm , we define M 2 Rdm as the tensor such that (M )j1,...,jm := Mj 1 ,...,j m 8j1, . . . , jm 2 [d]. In other words M is a copy of M with its axes permuted. This generalizes the concept of transpose to tensors. Let Pm be the set of all permutations of {1, . . . ,m}. We say that a tensor X 2 Rdm is symmetric if and only if X = X 8 2 Pm. We denote the set of symmetric tensors by Sdm . Given M 2 Rdm , we define the symmetrization of M by S(M) = 1 m! X 2Pm M . Note that when m = 2, then S(M) = 2 (M +M). Given x 2 Rd, we define a symmetric rank-one tensor by x⌦m := x⌦ · · ·⌦ x | {z } m times 2 Sdm , i.e., (x⌦m)j1,j2,...,jm = xj1xj2 . . . xjm . We denote the symmetric outer product decomposition (Comon et al., 2008) of W 2 Sd m by